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Claims 

1 . (Currently Amended) A method of constructing a model operable to 
generate one or more job performance criteria predictions based on input pre-hire 
information, the method comprising: 

electronically collecting pre-hire information from a plurality of applicants 
wherein at least some of the pre-hire information is collected from at least one of the 
applicants who responds directly on an electronic device to provide pre-hire applicant 
responses to questions : 

collecting post-hire information for the applicants based on job performance of the 
applicants after hire^ ^ wherein collecting post - liire information compris es r e ceiving 
payroll information for the applicants via a nctworlc, d e t e rmining a t e rmination 
date for an applicant from the payroll information received via the nctworlc, and 
determining tenure of the applicant by comparing the termination date with a 
hiring date of the applicant; and 

via information-theoretic feature selection, choosing questions from the pre- 
hire information as features for which respective pre-hire applicant responses serve 
as inputs to the model, wherein the information-theoretic feature selection 
comprises identifying at least one higher-order interaction comprising a set of a 
pluralitv of questions having higher predictive power than a sum of predictive 
powers of individual questions in the set; and 
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from the pre-hire information and the post-hire information, generating training 
an artificial intelligence-based predictive model in a computer-readable medium with 
observed pre-hire applicant responses for the chosen features , wherein the artificial 
intelligence-based predictive model is operable to generate one or more job performance 
criteria predictions comprising tenure based at least on input pre-hire information from 
new applicants corresponding to the chosen features , whereby the one or more job 
performance criteria predictions are usable as a basis for a hiring recommendation or 
other employee selection information. 

2. (Currently Amended) A computer-readable medium comprising 
computer-executable instructions for performing a method of constructing a model 
operable to generate one or more job performance criteria predictions based on input pre- 
hire information, the method comprising: 

electronically collecting pre-hire information from a plurality of applicants 
wherein at least some of the pre-hire information is collected from at least one of the 
applicants who responds directly on an electronic device to provide pre-hire 
applicant responses to questions : 

collecting post-hire information for the applicants based on job performance of the 
applicants after hirei , wh e r e in coll e cting post hir e information compris e s r e c e iving 
payroll information for th e applicants via a n e twork, d e t e rmining a t e rmination dat e for 
on applicant from th e payroll information r e c e iv e d via th e network, and d e t e rminin g 
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t e nur e of th e applicant by comparing tho termination dat e with a hiring dat e of the 
appUoant; and 

via information-theoretic feature selection, choosing questions from the pre- 
hire information as features for which respective pre-hire applicant responses serve 
as inputs to the model, wherein the information-theoretic feature selection 
comprises identifying at least one higher-order interaction comprising a set of a 
pluralitv of questions having higher predictive power than a sum of predictive 
powers of individual questions in the set; and 

from the pre-hire information and the post-hire information, gcncroting training 
an artificial intelligence-based predictive model with observed pre-hire applicant 
responses for the chosen features, wherein the artificial intelligence-based predictive 
model is operable to generate one or more job performance criteria predictions 
comprising tenure based at least on input pre-hire information from new appUcants 
corresponding to the chosen features . 



3. (Canceled) 



4. (Canceled) 
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5. (Original) The method of claim 1 further comprising: 

limiting the applicants for the model to those with a particular occupation; and 
constructing the model as an occupationally-specialized model. 

6. (Original) The method of claim 1 wherein the model accepts one or more 
inputs, the method further comprising: 

identifying in the pre-hire information one or more characteristics that are 
ineffective predictors; and 

omitting the ineffective predictors as inputs to the model. 

7. (Original) The method of claim 1 wherein the pre-hire information 
comprises one or more characteristics, the method further comprising: 

identifying in the pre-hire information one or more characteristics that are 
ineffective predictors; and 

providing an indication that the characteristics no longer need to be collected. 

8. (Previously Presented) The method of claim 1 wherein job performance 
criteria predictions comprise a prediction indicating whether a job candidate will be 
voluntarily terminated. 
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9. (Previously Presented) The method of claim 1 wherein job performance 
criteria predictions comprise a prediction indicating whether a job candidate will be 
eligible for rehire after termination. 

10. (Previously Presented) The method of claim 1 wherein the pre-hire 
information comprises one or more characteristics, the method further comprising: 

identifying in the pre-hire information one or more characteristics that are 
ineffective predictors; 

responsive to identifying the ineffective predictors, collecting new pre-hire 
information not including the ineffective predictors; and 

building a refined model based on the new pre-hire information. 

1 1 . (Original) The method of claim 1 0 further comprising: 

adding one or more new characteristics to be collected when collecting the new 
pre-hire information. 

12. (Original) The method of claim 1 1 further comprising: 
evaluating the effectiveness of the new characteristics. 

13. (Canceled) 
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14. (Canceled) 

15. (Currently Amended) A method for constructing an artificial 
intelligence-based employment selection process based on pre-hire information 
comprising personal employee characteristics and post-hire information comprising 
employee job performance observation information, the method comprising: 

via information-theoretic feature selection, choosing questions from the pre- 
hire information as chosen features for which respective pre-hire applicant 
responses serve as inputs to predictive artificial models, wherein the information- 
theoretic feature selection comprises identifying at least one higher-order 
interaction comprising a set of a pluralitv of questions having higher predictive 
power than a sum of predictive powers of individual questions in the set; 

generating a plurality of predictive artificial intelligence models accepting the 
chosen features as inputs, b ased on the pre-hire and post-hire information, wherein at 
least two of the artificial intelligence models are of different feed-forward types and 
different architectures; 

testing effectiveness of the models to select an effective model; and 

applying the effective model to predict post-hire information not yet observed, 
whereby the post-hire information not yet observed that is predicted by the effective 
model can be a basis for a hiring recommendation or other employee selection 
information. 
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16. (Canceled) 

17. (Previously Presented) The method of claim 15 wherein at least one of the 
models is an expert system. 

18. (Canceled) 

19. (Canceled) 

20. (Canceled) 

21. (Original) The method of claim 15 further comprising: 
identifying at least one of the models as exhibiting impermissible bias; and 
avoiding use of the models exhibiting impermissible bias. 

22. (Canceled) 
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23. (Currently Amended) A computer-implemented method of refining an 
artificial-intelligence based employee performance selection system, the method 
comprising: 

collecting information via an electronic device presenting a set of questions to 
employment candidates, wherein the questions are stored in a computer-readable 
medium; 

collecting post-hire information for the employment candidates based on job 
performance of the employment candidates after hire, wherein collecting post-hire 
information comprises receiving payroll information for the employment candidates via a 
network, determining a termination date for an employment candidate from the payroll 
information received via the network, and determining tenure of the employment 
candidate by comparing the termination date with a hiring date of the applicant; 

testing effectiveness of at least one of the questions in predicting tenure ^ wherein 
testing effectiveness comprises considering varions sets of permutations of 
predictive items comprising tlie at least one of tlie questions, wherein tlie at least one 
of the questions alone is ineffective but is effective in combination with one or more 
other items : and 

responsive to determining the question is ineffective in predicting tenure, deleting 
the question from the computer-readable medium. 

24. (Canceled) 
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25. (Currently Amended) A computer-readable medium comprising a 
predictive model, the model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for 
a job applican t wherein at least one of the inputs is chosen via information-theoretic 
feature selection and corresponds to a question for which pre-hire applicant 
responses have been collected^ wherein the information-theoretic feature selection 
comprises identifying at least one higher-order interaction comprising a set of a 
plurality of questions having higher predictive power than a sum of predictive 
powers of individual questions in the set : and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs, 

wherein the predictive model is an artificial intelligence-based model constructed 
from pre-hire data electronically collected from a plurality of employees and post-hire 
data, and the model generates its predictive outputs based on the similarity of the inputs 
to pre-hire data collected for the plurality of employees and their respective post-hire 
data; and 

wherein the one or more predicted job effectiveness criteria comprise at least one 
selected fi*om the group consisting of: 

whether the job applicant will be involuntarily terminated, and whether the job 
applicant will be eligible for rehire upon termination. 
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26. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating a rank for the job applicant. 

27. (Original) The computer-readable medium of claim 26 wherein the rank is 
relative to other applicants. 

28. (Original) The computer-readable medium of claim 26 wherein the rank is 
relative to the plurality of employees. 

29. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating probability of group 
membership for the job applicant. 

30. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating predicted tenure for the job 
applicant. 

31. (Canceled) 
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32. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating predicted number of accidents 
for the job applicant. 

33. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating whether the applicant will be 
involuntarily terminated. 

34. (Original) The computer-readable medium of claim 25 wherein the 
predictive model comprises a predictive output indicating whether the applicant will be 
eligible for rehire after termination, 

35. (Canceled) 

36. (Canceled) 
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37. (Previously Presented) A computer-readable medium comprising a refined 
predictive artificial intelligence-based model, the model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for 
a job applican t wherein at least one of the inputs is chosen via information-theoretic 
feature selection and corresponds to a question for which pre-hire applicant 
responses have been collected, wherein the information-theoretic feature selection 
comprises identifying at least one higher-order interaction comprising a set of a 
pluralitv of questions having higher predictive power than a sum of predictive 
powers of individual questions in the set : and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs; 

wherein the one or more predictive outputs are configured to predict whether the 
job applicant will be involuntarily terminated or whether the job applicant will be eligible 
for rehire upon termination, and 

wherein the predictive model is constructed from pre-hire data electronically 
collected from a plurality of employees and post-hire data, wherein the pre-hire data is 
based on a question set refined by having identified and removed one or more questions 
as ineffective. 



38. (Canceled) 
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39. (Canceled) 

40. (Currently Amended) An apparatus for assisting in determining the 
suitability of an individual for employment by an employer, the apparatus comprising: 

a computer-readable medium comprising a predictive model comprising: 
inputs for accepting one or more characteristics based on pre-hire information for 
a job applican t wherein at least one of the inputs is chosen via information-theoretic 
feature selection and corresponds to a question for which pre-hire applicant 
responses have been collected, wherein the information-theoretic feature selection 
comprises identifying at least one higher-order interaction comprising a set of a 
pluralitv of questions having higher predictive power than a sum of predictive 
powers of individual questions in the set : and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs, wherein the predictive model is an artificial intelligence- 
based model constructed from pre-hire data electronically collected from a plurality of 
employees and post-hire data, and the model generates its predictive outputs based on the 
similarity of the inputs to pre-hire data collected for the plurality of employees and their 
respective post-hire data; wherein the one or more predicted job effectiveness criteria 
comprise at least one selected from the group consisting of: whether the job applicant will 
be involuntarily terminated, and whether the job applicant will be eligible for rehire upon 
termination; 
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electronic data interrogator means for presenting a first set of a plurality of means 
for questioning to the individual; 

electronic answer capturer means for electronically storing answers by the 
individual to at least a selected plurality of the first set of means for questioning 
presented to the individual; 

electronic predictor means responsive to the stored answers and for predicting at 
least one post-hire outcome if the individual were to be employed by the employer, the 
predictor providing a prediction of the outcome based upon correlations of the stored 
answers with answers to sets of means for questioning by other individuals for which 
post-hire information has been collected; 

electronic results provider means for providing an output indicative of the 
outcome to assist in determining the suitability of the individual for employment by the 
employer. 

41 . (Currently Amended) An apparatus for assisting in determining the 
suitability of an individual for employment by an employer, the apparatus comprising: 

an electronic device for presenting a first set of a plurality of means for 
questioning to the individual; 

means for electronically storing answers by the individual to at least a selected 
plurality of the first set of means for questioning presented to the individual; 
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means responsive to the stored answers and for predicting at least one post-hire 
outcome if the individual were to be employed by the employer, the predictor providing a 
prediction of the outcome based upon correlations of the stored answers with answers to 
sets of means for questioning by other individuals for which post-hire information has 
been collected; and 

means for providing an output indicative of the outcome to assist in determining 
the suitability of the individual for employment by the employer, 

wherein the apparatus comprises a predictive model comprising: 

inputs for accepting one or more characteristics based on pre-hire information for 
a job apDlican t wherein at least one of the inputs is chosen via information-theoretic 
feature selection and corresponds to a question for which pre-hire applicant 
responses have been collected, wherein the information-theoretic feature selection 
comprises identifying at least one higher-order interaction comprising a set of a 
plurality of questions having higher predictive power than a sum of predictive 
powers of individual questions in the set : and 

one or more predictive outputs indicating one or more predicted job effectiveness 
criteria based on the inputs. 
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wherein the predictive model is an artificial intelligence-based model constructed 
from pre-hire data electronically collected from a plurality of employees and post-hire 
data, and the model generates its predictive outputs based on the similarity of the inputs 
to pre-hire data collected for the plurality of employees and their respective post-hire 
data; and 

wherein the one or more predicted job effectiveness criteria comprise at least one 
selected from the group consisting of: 

whether the job applicant will be involuntarily terminated, and whether the job 
applicant will be eligible for rehire upon termination. 

42. (Canceled) 

43. (Previously Presented) The computer-readable medium of claim 37 
wherein the ineffective questions are identified via an information transfer technique 
comprising: 

considering various sets of permutations of answers to questions; 
evaluating whether the sets of permutations are effective; and 
determining whether an answer to a question is not in any effective set of 
permutations. 
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44. (Currently Amended) The method of claim i 46 wherein collecting post- 
hire information further comprises: 

tracking whether the applicant has been dropped from payroll 

45. (Previously Presented) The method of claim 2 1 wherein: 

the identifying comprises comparing across gender, ethnicity, or age differences. 

46. (New) The method of claim 1 wherein: 

collecting post-hire information comprises receiving payroll information for the 
applicants via a network, determining a termination date for an applicant from the payroll 
information received via the network, and determining tenure of the applicant by 
comparing the termination date with a hiring date of the applicant. 

47. (New) The method of claim 1 wherein: 

the set of the plurality of questions having higher predictive power than the sum of 
predictive powers of individual questions in the set comprises at least one biodata 
question and at least one psychometric question. 
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48. (New) The method of claim 1 wherein identifying at least one higher-order 
interaction comprises: 

generating an approximation of an optimal subset of quetsions for use as input 
features for the model. 

49. (New) The method of claim 48 wherein generating an approximation of an 
optimal subset of questions comprises: 

determining a union of a set of questions appearing in predictive transmissions of 
greatest magnitude. 

50. (New) The method of claim 48 wherein generating an approximation of an 
optimal subset of questions comprises: 

from a set of transmissions T^, choosing m unique transmissions of greatest 
magnitude as a base set for higher-order transmissions; 

generating T'k+i, by adding questions to members of Tk, that generate a set T'k+i 
with largest transmission values; and 

taking a union of questions appearing in the unique transmissions of greatest 
magnitude, wherein the union approximates the optimal subset of questions. 
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